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(ENTRY INTO U.S. NATIONAL PHASE UNDER CHAPTER H) 

PCT/US99/04206 25/Feb/1999 25/Feb/1998 

INTERNATIONAL APPLICATION NO. INTERNATIONAL FILING DATE PRIORITY DATE CLAIMED 

Data Integration and Registration Method and Apparatus for Non-Destructive Evaluation of Materials 
TITLE OF INVENTION 

Steven M. Shepard 

APPLICANTS) 
Box PCT 

Assistant Commissioner for Patents 
Washington D.C. 20231 

ATTENTION: EO/US 

NOTE: To avoid abandonment of the application, the applicant shall furnish to the USPTO, not later than 20 months from the 
priority date: (I) a copy of the international application, unless it has been previously communicated by the International 
Bureau or unless it was originally fded in the USPTO; and (2) the basic national fee (see 37 C.F.R. § 1.492(a)). The 30- 
month time limit may not be extended. 37 C.F.R. § 1.495. 

WARNING: Where the items are those which can be submitted to complete the entry of the international application into the 



CERTIFICATION UNDER 37 C.F.R. 1.10* 

(Express Mail label number is mandatory.) 
(Express Mail certification is optional) 

I hereby certify that this correspondence and ^documents referred to as attached therein are being deposited with the United 

States Postal Service on this date 08/25/00 , in an envelope as "Express Mail Post Office to Addressee," 

Mailing Label Number EL686848486US addressed to the: Assistant Commissioner for Patents, Washington, D.C. 20231 . 

{type or print name of person mailing paper) 

WARNING: Certificate of mailing (first class) or facsimile transmission procedures of 37 C.F.R. L8 cannot be used to 

obtain a date of mailing or transmission for this correspondence. 

* WARNING: Each paper or fee filed by "Express Mail " must have the number of the "Express Mail " mailing label 
placed thereon prior to mailing. 37 C.F.R. 1.10(b). 

"Since the filing of correspondence under §1.10 without the Express Mail mailing label thereon is an 
oversight that can be avoided by the exercise of reasonable care, requests for waiver of this requirement will 
not be granted on petition. " Notice of Oct. 24, 1996, 60 Fed. Reg. 56,439, at 56,442. 
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national phase are subsequent to 30 months from the priority date the application is still considered to be in the 
international state and if mailing procedures are utilized to obtain a date the express mail procedure of 37 
CF.R. §1.10 must be used (since international application papers are not covered by an ordinary certificate of 
mailing - See 37 C.F R. §1.8. 

NOTE: Documents and fees must be clearly identified as a submission to enter the national state under 35 USC 371 otherwise the 
submission will be considered as being made under 35 USC 111. 37 CF.R. § 1.494(f). 



1. Applicant herewith submits to the United States Elected Office (EO/US) the following items under 
35U.S.C. 371: 

a. , [X] This express request to immediately begin national examination procedures (35 

U.S.C. 371(f)). 

b. [X] The U.S. National Fee (35 U.S.C. 371(c)(1)) and other fees (37 CF.R. § 1.492) as 
^ indicated below: 
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2. Fees 



CLAIMS 
FEE 


(l)FOR 


(2) NUMBER 
FILED 


(3) NUMBER 
EXTRA 


(4) RATE 


(5) CALCULA- 1 
TIONS 


[]* 


TOTAL 
CLAIMS 


28 -20 = 


8 


x $18.00 = 


$144.00 




INDEPENDENT 
CLAIMS 


5 -3 = 


2 


x $78.00 = 


156,00 




MULTIPLE DEPENDENT CLAIM(S) (if applicable) + S260.00 




BASIC FEE** 


[ x ] U.S. PTO WAS INTERNATIONAL PRELIMINARY EXAMINATION 
AUTHORITY 

Where an International preliminary examination fee as set forth in § 1 .482 
has been paid on the international application to the U.S. PTO: 
[ ] and the international preliminary examination report states that 
the criteria of novelty, inventive step (non-obviousness) and 
industrial activity, as defined in PCT Article 33(2) to (4) have 
been satisfied for all the claims presented in the application 

entering the national stage (37 CFR 1.492(a)(4)) S96.00 

[ x ] and the above requirements are not met (37 CFR 1 .492(a)(1)) 
$670.00 


$670.00 




[ ] U.S. PTO WAS NOT INTERNATIONAL PRELIMINARY 

t-ixt i t f t\t A TTAXT A T T' I'T T/"VT* TT\7 

EXAMINATION AUTHORITY 

Where no international preliminary examination fee as set forth 
in § 1 .482 has been paid to the U.S. PTO, and payment of an 
international search fee as set forth in § 1.445(a)(2) to the U.S. 
PTO: 

[ ] has been paid (37 CFR 1 .492(a)(2)) $760.00 

[ ] has not been paid (37 CFR 1 .492(a)(3)) $970.00 

[ ] where a search report on the international application has been 
prepared by the European Patent Office or the Japanese Patent 
Office (37 CFR 1.492(a)(5)) $840.00 






Total of above Calculations 


- 970.00 


SMALL 
ENTITY 


Reduction by Vi for filing by small entity, if applicable. Affidavit must be filed, (note 
37 CFR 1.9, 1.27, 1.28) 


-$485.00 




Subtotal 


$485.00 




Total National Fee 


$ 485.00 




Fee for recording the enclosed assignment document $40.00 (37 CFR 1.21(h)). (See 
Item 13 below). See attached "ASSIGNMENT COVER SHEET". 




TOTAL 


Total Fees enclosed 


$ 485.00 — 
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*See attached Preliminary Amendment Reducing the Number of Claims. 

i. [ ] A check in the amount of to cover the above fees is enclosed. 

ii. [X] Please charge Account No. 18-0013 in the amount of $ 485.00 . 
A duplicate copy of this sheet is enclosed. 

* WARNING: "To avoid abandonment of the application the applicant shall furnish to the United States Patent and 
Trademark Office not later than the expiration of 30 months from the priority date: * * * (2) the basic 
national fee (see § 1.492(a)). The 30-month time limit may not be extended. "37 C.F.R. § 1.495(b). 

WARNING: If the translation of the international application and/or the oath or declaration have not been submitted by 

the applicant within thirty (30) months from the priority date, such requirements may be met within a time 
period set by the Office. 37 C.F.R. § 1.495(b)(2). The payment of the surcharge set forth in § 1.492(e) is 
required as a condition for accepting the oath or declaration later than thirty (30) months after the priority 
date. The payment of the processing fee set forth in § 1.492(f) is required for acceptance of an English 
translation later than thirty (30) months after the priority date. Failure to comply with these requirements 
will result in abandonment of the application. The provisions of§L 136 apply to the period which is set. 
Notice of Jan. 3, 1993, 1147 O.G. 29 to 40. 

3. [X] A copy of the International application as filed (35 U.S.C. 371(c)(2)): 

NOTE: Section 1.495 (b) was amended to require that the basic national fee and a copy of the international application must 
be filed with the Office by 30 months from the priority date to avoid abandonment 'The International Bureau normally 
provides the copy of the international application to the Office in accordance with PCT Article 20. At the same time, 
the International Bureau notifies applicant of the communication to the Office. In accordance with PCT Rule 47.1, 
that notice shall be accepted by all designated offices as conclusive evidence that the communication has duly taken 
place. Thus, if the applicant desires to enter the national stage, the applicant normally need only check to be sure the 
notice from the International Bureau has been received and then pay the basic national fee by 30 months from the 
priority date. " Notice of Jan. 7, 1993, 1147 O.G. 29 to 40, at 35-36. See item 14c below. 

a. [X] is transmitted herewith. 

b. [ ] is not required, as the application was filed with the United States Receiving 

Office. 

c. [ ] has been transmitted 

i. [ ] by the International Bureau. 

Date of mailing of the application (from form PCTVIB/308): . 

ii. [ ] by applicant on . 

Date 

4. [X] A translation of the International application into the English language (35 U.S.C. 

371(c)(2)): 

a. [ ] is transmitted herewith. 

b. [ x ] is not required as the application was filed in English. 

c. [ ] was previously transmitted by applicant on . 

Date 

d. [ ] will follow. 



[ ] Amendments to the claims of the International application under PCT Article 19 (35 
U.S.C. 371(c)(3)): 
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NOTE: The Notice of January 7, 1993 points out that 37 CF.R. § 1.495(a) was amended to clarify the existing and continuing 
practice that PCT Article 19 amendments must be submitted by 30 months from the priority date and this deadline may 
not be extended. The Notice further advises that: "The failure to do so will not result in loss of the subject matter of the 
PCT Article 19 amendments. Applicant may submit that subject matter in a preliminary amendment filed under section 
1.121. In many cases, filing an amendment under section 1.121 is preferable since grammatical or idiomatic errors 
may be corrected. "1147 O.G. 29-40, at 36. 



[ ] are transmitted herewith. 

;en transmitted 
by the International Bureau, 
'mailing of the amendment (fi 
by applicant on . 



b. 


[ ] 


have 




i. 


t ] 






Date 




ii. 


[ ] 


c. 


[ ] 


have 




i. 


[ ] 






Date 




ii. 


[ ] 



Date 

)t been transmitted as 

applicant chose not to make amendments under PCT Article 19. 



the time limit for the submission of amendments has not yet expired. The 
amendments or a statement that amendments have not been made will be 
transmitted before the expiration of the time limit under PCT Rule 46. 1 . 

6. [ ] A translation of the amendments to the claims under PCT Article 19 (38 U.S.C. 

371(c)(3)): 

a. [ ] is transmitted herewith. 

b. [ ] is not required as the amendments were made in the English language. 

c. [ ] has not been transmitted for reasons indicated at point 5(c) above. 

7. [ x ] A copy of the international examination report (PCT/IPEA/409) 
— ~ [ x ] is transmitted herewith. 

[ ] is not required as the application was filed with the United States Receiving 
Office. 

8. X x ] Annex(es) to the international preliminary examination report 

a. [ ] is/are transmitted herewith. 

b. [ x ] is/are not required as the application was filed with the United States Receiving 

Office. 

9. [ ] A translation of the annexes to the international preliminary examination report 

a. [ ] is transmitted herewith. 

b. [ ] is not required as the annexes are in the English language. 

10. [X] An oath or declaration of the inventor (35 U.S.C. 371(c)(4)) complying with 35 U.S.C. 

115 

a. [ ] was previously submitted by applicant on . 

Date 

b. [ ] is submitted herewith, and such oath or declaration 

i. [ ] is attached to the application. 

ii. [ ] identifies the application and any amendments under PCT Article 19 that 

were transmitted as stated in points 3(b) or 3(c) and 5(b); and states that 
they were reviewed by the inventor as required by 37 CF.R. 1 .70. 
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iii. [X] will follow. 
Other document(s)'or information included: 

11. [x] An International Search Report (PCT/ISA/2 10) or Declaration under PCT Article 

17(2)(a): 

a. [ ] is transmitted herewith. 

b. [ ] has been transmitted by the International Bureau. 

Date of mailing (from form PCT/IB/308): . 

c. [ x ] is not required, as the application was searched by the United States 

International Searching Authority. 

d. [ ] will be transmitted promptly upon request. 

e. [ ] has been submitted by applicant on . 

Date 

12. [X] An Information Disclosure Statement under 37 C.F.R. 1.97 and 1.98: 

a. [ ] is transmitted herewith. 

Also transmitted herewith is/are: 
[ ] Form PTO-1449 (PTO/SB/08A and 08B). 
[ ] Copies of citations listed. 

b. [ x ] will be transmitted within THREE MONTHS of the date of submission of 

requirements under 35 U.S.C. 371(c). 

c. [ ] was previously submitted by applicant on . 

Date 

13. [ ] An assignment document is transmitted herewith for recording. 

A separate [ ] "COVER SHEET FOR ASSIGNMENT (DOCUMENT) ACCOMPANYING 

NEW PATENT APPLICATION" or[ ] FORM PTO 
1595 is also attached. 



14. [X] Additional documents: 

a. [ ] Copy of request (PCT/RO/101) 

b. [ x ] International Publication No. WQ99/44366 

i. [ ] Specification, claims and drawing 

ii. [ x ] Front page only 

c. [ ] Preliminary amendment (37 C.F.R. § 1.121) 

d. [ ] Other 



15. 



[X] 



The above checked items are being transmitted 
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a. [X] before 30 months from any claimed priority date. 

b. " [ ] after 30 months. 

16. [ ] Certain requirements under 35 U.S.C. 371 were previously submitted by the applicant on 
, namely: 



AUTHORIZATION TO CHARGE ADDITIONAL FEES 

WARNING: Accurately count claims, especially multiple dependent claims, to avoid unexpected high charges if extra 
claims are authorized. 

NOTE: "A written request may be submitted in an application that is an authorization to treat any concurrent or future reply, 
requiring a petition for an extension of time under this paragraph for its timely submission, as incorporating a petition 
for extension of time for the appropriate length of time. An authorization to charge all required fees, fees under § 1.17, 
or all required extension of time fees will be treated as a constructive petition for an extension of time in any 
concurrent or future reply requiring a petition for an extension of time under this paragraph for its timely submission. 
Submission of the fee set forth in § LI 7(a) will also be treated as a constructive petition for an extension of time in any 
concurrent reply requiring a petition for an extension of time under this paragraph for its timely submission. "37 
C.F.R. § 1.136(a)(3). 

NOTE' "Amounts of twenty-five dollars or less will not be returned unless specifically requested within a reasonable time, nor 
will the payer be notified of such amounts; amounts over twenty-five dollars may be returned by check or, if requested, 
by credit to a deposit account. "37 C.F.R. § 1.26(a). 

^^X\ The Commissioner is hereby authorized to charge the following additional fees that may 
be required by this paper and during the entire pendency of this application to Account 
No. 18-0013. 



^ . - [X] 37 C.F.R. 1 .492(a)(1), (2), (3), and (4) (filing fees) 

WARNING: Because failure to pay the national fee within 30 months without extension (37 C.F.R. § 1.495(b)(2)) results 
in abandonment of the application, it would be best to always check the above box. 

[X] 37 C.F.R. 1 .492(b), (c) and (d) (presentation of extra claims) 



NOTE: Because additional fees for excess or multiple dependent claims not paid on filing or on later presentation must only 
be paid or these claims cancelled by amendment prior to the expiration of the time period set for response by the PTO 
in any notice of fee deficiency (37 C.F.R. § 1.492(d)), it might be best not to authorize the PTO to charge additional 
claim fees, except possible when dealing with amendments after final action. 



[X] 37 C.F.R. 1.17 (application processing fees) 
[X] 37 C.F.R. 1.17(a)(l)-(5)(extension fees pursuant to § 1.136(a). 
[ ] 37 C.F.R. 1.18 (issue fee at or before mailing of Notice of Allowance, pursuant 
to37C.F.R. 1.311(b)) 



NOTE: Where an authorization to charge the issue fee to a deposit account has been filed before the mailing of a Notice of 
Allowance, the issue fee will be automatically charged to the deposit account at the time of mailing the notice of 
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allowance. 37 C.F.R. § 1311(b). 

NOTE: 37 C.F.R. 1.28(b) requires "Notification of any change in loss of entitlement to small entity status must be filed in the 
application . . . prior to paying, or at the time of paying . , . issue fee. " From the wording of 37 C.F.R. § 1.28(b): (a) 
notification of change of status must be made even if the fee is paid as "other than a small entity" and (b) no 
notification is required if the change is to another small entity. 

[X] 37 C.F.R. § 1.492(e) and (f) (surcharge fees for filing the declaration and/or 

filing an English translation of an International Application later than 30 months 




P.O. Address 

39533 Woodward Ave., Suite 140 
Bloomfield Hills. MI 48304 



CUSTOMER NO.: 010291 
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D ATA INT-EGRATJO^NDREGISTRATION METHOD AN^APPARATUS 



FOR NON-DESTRUCTIVE EVALUATION OF MATERIALS 



TECHNICAL FIELD 



5 



The present invention is directed to a method and apparatus for non-destructive 



testing and evaluation of materials, and more particularly to a method and apparatus for 
identifying and registering defects in a sample by superimposing a defect image over a 
live image to locate subsurface defects in the sample. 



Various methods of non-destructive testing and evaluation (NDT/E) of parts have 
been developed to detect subsurface defects in a part sample and to measure the depth of 
subsurface defects. These methods include step thermography, pulse thermography, 
pulse thermography, and other thermographic techniques. All of these techniques 

15 involve deliberately changing the temperature of the sample, allowing the sample to 
return to equilibrium temperature, and observing the temperature change of the sample 
via an infrared camera. Anomalous temperature changes that appear in the infrared 
camera image indicate subsurface defects in the sample; subsurface defects tend to 
impede the normal heat flow in the sample and will appear as anomalies in the image. 

20 Further, because the infrared image showing the defect is transient and may last for only 
a fraction of a second, the image must be captured (usually with a digital computer) and 
then verified with the actual sample to locate the defect. 

The actual verification process, usually through a complementary NDT/E 
process, can be relatively difficult because the infrared defect image of the sample may 

25 bear little resemblance to the actual sample. For example, many subsurface defects 
appear only in the infrared image; to the naked eye, the sample containing the defects 
often appears perfectly uniform. As a result, a user must attempt to match the image of 
the subsurface defect with the actual, unblemished sample surface to pinpoint the 
location of the defect. This is further complicated by the fact that the infrared camera 
30 lens often distorts the image, causing straight lines at the periphery of the lens's field of 
view to appear curved in the image. To locate and mark the positions of subsurface 



10 



BACKGROUND ART 



l 



defects with some precision, prior art methods include using regularly spaced registration 
markers on the sample, calculating complex anamorphic mapping algorithms, or printing 
a full-size defect image and physically matching or overlaying the full-size image onto 
the actual sample. Because the sample may not have any distinguishing marks that 
5 appear in the defect image, precise registration of the image and the sample's surface can 
be difficult. In addition, these methods are time-consuming and are not particularly 
convenient, and at best they can only approximate the subsurface defect location due to 
the image distortion from the infrared camera lens. Further, measuring the depth of 
subsurface defects often requires some prior knowledge of the sample's dimensions or 
10 properties, such as the thickness of the sample, the depth of a known defect, the 

material's thermal diffusivity, etc. This information is often not available in practice, 
making precise depth measurements difficult with known techniques. 

Thus, there currently is a need for a NDT/E technique that allows accurate 
annotation, marking, and thickness measurements of specific locations on a sample, 
15 without the problems caused by differences between the image and the actual sample due 
to image distortion. There is also a need for a NDT/E technique that can conduct depth 
measurement without requiring prior knowledge of any of the sample's characteristics. If 
the depth and onset time (relative to the heating event) of a single defect are known, it is 
then possible to calculate the thermal diffusivity of the material and to use it to determine 
20 the depths of other defects from their offset time, according to the well-known relation of 
t = d 2 / D, where t is the onset time, d is the depth of the defect, and D is the thermal 
diffusivity of the sample. 

SUMMARY OF THE INVENTION 
25 Accordingly, the present invention is a method and apparatus for conducting 

NDT/E that simplifies the correspondence between image information obtained during 
NDT/E and the actual sample. More specifically, the invention involves obtaining a 
defect image and a live image of the same sample and then superimposing one image on 
the other. One embodiment of the invention is directed to linking the information 
30 regarding surface defects obtained during infrared NDT/E to the actual part being 

inspected. The invention includes generating a defect image of the sample via infrared 



imaging or some other means. The defect image may have markers or other indicia 
locating where subsurface defects are in the sample. The defect image is then 
superimposed onto a live image of the sample. A user then views the live image of the 
sample, rather than the sample itself, while transferring the marks from the defect image 
to the sample. Because both the defect image and the live image are distorted by the 
infrared camera lens and therefore have a one-to-one correspondence, the distorted image 
is used as the frame of reference for locating subsurface defects and marking the sample. 
This ensures that the marks in the defect image are transferred precisely from the defect 
image onto the sample and also eliminates the need to map the distorted image to the 
sample in a separate step. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Figure 1 is a flowchart illustrating an evaluation and testing method according to 

the present invention; 

Figure 2 is a flowchart illustrating an example of the inventive method as applied 

to thermography; 

Figures 3 a through 3f illustrate actual images taken from infrared NDT/E of an 
aluminum aircraft panel according to the method of the present invention; and 

Figure 4 is a representative diagram of an apparatus that can be used to annotate, 
calibrate, and/or evaluate a sample according to the method of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
Referring to Figure 1, one embodiment of the NDT/E method according to the 
present invention can be broken down into four steps. First, a defect image of the sample 
is obtained 100, digitized and displayed on a computer 102 using a computer program 
that has a referencing mechanism, such as drawing tool that allows the user to draw on 
the'defect image using a mouse, touch screen, light pen, or other pointing device. The 
user can mark defects found in the defect image with the drawing tool 103. The defect 
image, including any marks made by the user, is then superimposed onto a live image 
104, which is also displayed on the computer. The live image is preferably produced 
immediately thereafter and using the same lens and camera that produced the defect 



image to ensure a one-to-one correspondence between the live image and the defect 
image; using the same lens ensures that both images will have the same distortion. The 
user then marks the actual sample, using a marking pencil or similar device, while 
viewing the live image 106 rather than looking the sample itself. Instead of marking the 
5 part, the user may also use a point measuring device to measure characteristics of the 
sample, such as its thickness, and append the data to the image for annotation or 
calibration purposes. The defect image and live image can also be simply superimposed 
one atop the other for referencing purposes, without any user intervention. 

As noted above, the defect image is subject to any lens distortion that may be 
10 present. In the prior art, inaccuracies in marking occurred because the user was 

attempting to map the distorted defect image onto a corresponding physical sample part, 
whether it was through markers on the sample or through overlaying a full-size image on 
the actual sample. In the invention, however, the user watches the live image of the 
sample rather than the sample itself when referencing the sample. Because the live 
15 image and the defect image in the present invention are both distorted via the same 
camera lens, the defect image and the live image align perfectly with each other. 
Further, the user will look at the live image of the marking or measurement instrument, 
and not the actual marking/measurement instrument itself, in real time while referencing 
the sample, making it possible to trace the marks from the defect image onto the sample 
20 precisely and distortion-free. In short, the distorted defect image and the distorted live 
image both have exactly the same, albeit distorted, frame of reference. Thus, any marks 
or measurements taken while viewing the live image will correspond exactly to the 
marked locations in the defect image superimposed on the live image. 

Referring now to Figures 2 and 3a through 3f, the method of the present 
25 invention will now be described with respect to thermography. Note that the invention is 
not limited to such an application and can be used in any application where accurate 
mapping between an image of a sample and the sample itself is needed. The first step 
involves changing the sample's temperature 200 by heating or cooling the temperature 
through any known method. For example, heat can be applied to the sample via a 
30 flashlamp or a continuous lighting source. Defect images of the sample are then taken 
202 with an infrared camera as the sample's temperature returns to equilibrium. The 

4 



infrared camera preferably obtains multiple defect images from the same sample after the 
sample's temperature is changed, taking images at selected time intervals as the sample 
returns to equilibrium temperature. Figures 3a through 3d illustrate a time sequence of 
infrared images of an aluminum aircraft pane adhesively bonded to an aluminum 
5 structural frame, illustrating the transient nature of infrared NDT/E. Figure 3a was taken 
just before heating, Figure 3b taken 1.33 seconds after heating, Figure 3c taken 10.67 
seconds after heating, and Figure 3d taken 39.34 seconds after heating. As can be seen 
in the figures, the aircraft panel does not exhibit any subsurface structure before heating 
(Figure 3a), while just after heating, the frame can be seen in the defect image, along 
10 with a disbonded area where corrosion has occurred, in the right half of the image 

(Figure 3b). The frame and disbonded area are less distinct after 10.67 seconds (Figure 
3c) and disappear completely after 3934 seconds (Figure 3d). For non-destructive 
evaluation, the image obtained shown in Figure 3b provides the most useful information 
because it is the clearest image. 
15 Once the desired defect image is selected, it is digitized and transferred to the 

computer 204 for display on a computer monitor. The defect image is preferably 
displayed using a program with a "draw" mode so that the user can place marks on the 
defect image using a mouse, touch screen, or other pointing device. The area where the 
defects occur is then "marked" on the screen by the user 206. An example of such a 
20 display is shown in Figure 3e, where the image from Figure 3b is taken as the defect 
image. As can be seen in Figure 3e, the area where the plate defect appears has been 
circled by the user, using the "draw" mode in the computer program generating the 
defect image, to highlight the area at which the subsurface defect is located. 

Once the defect image is marked 206, it is superimposed onto a digitized live 
25 image 208, which is also displayed on the computer monitor. As explained above, 

because both the defect image and the live image are obtained using the same infrared 
camera lens and because the camera is not moved after the defect image is obtained, both 
the defect image and the live image exhibit the same amount of lens distortion and have 
a one-to-one correspondence with each other; the defect image and live image do not 
30 have any distortion with respect to each other. Once the defect image is superimposed 
on the live image, the user can transfer the reference marks from the defect image to the 

5 



sample 210 exactly and distortion-free. As shown in Figure 3f, the user does this by 
viewing his hand (or the marking instrument) on the sample via the live image, not by 
directly watching the physical sample. Because of the one-to-one correspondence 
between the defect image and the live image, and because the user views the live image 
5 while referencing the actual sample, the marks are made on the actual sample precisely 
correspond to the location of the defect. Alternatively, or in addition to the marking, the 
user may also contact the sample with a point measuring probe, such as an ultrasonic 
thickness gauge or thermocouple, to measure the local properties of the sample. Because 
the probe will appear in the live image, the user can mark the position of the probe on the 
10 defect image and then read the probe measurement through a serial data connection. The 
information can be appended to the image for annotation or calibration purposes. 

Many different apparatus configurations can be used to carry out the method of 
the present invention. For example, although the above method has been described using 
a digital computer for receiving image data from the infrared camera, many infrared 
15 cameras have onboard displays and dedicated microcontrollers. Thus, most if not all of 
the steps explained above can be integrated into the camera itself. The user can perform 
the outlining/marking operation using cursor keys already resident on the camera. 
Further, the marking/annotation steps can be conducted using image processing/machine 
vision methods rather than manually by the user. Also, all of the above steps can be 
20 completely automated if, for example, a 3-D map of the sample is stored in the computer. 
In such a case, the computer itself could conduct the superimposing, referencing, and 
correlating based on the information in the 3-D map, without user intervention. 

Figure 4 illustrates one apparatus that can be used to carry out the inventive 
method with respect to thermography. The apparatus includes a reflective hood 400 that 
25 focuses light from heating lamps 402 onto a sample 404. The heating lamps 402 can be 
flashlamps (e.g. xenon lamps) or continuous lamps (e.g. halogen lamps). Regardless of 
the type of lamp used, reflectors 403 or some other means should be used to distribute 
light uniformly over the sample 404 surface at the opinion of the hood 400 so that the 
sample heats evenly. The front end of the hood 400 is open and is placed on or near the 
30 sample 404. The back portion of the hood 400 is designed to accommodate a lens 406 of 
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an infrared camera 408. The hood 400 also preferably has an access door 410 to allow 
the user to reach inside the hood 400 and mark the sample, if desired. 

The apparatus shown in Figure 4 also includes a display 412 for displaying the 
defect image and the live image generated by the infrared camera 408. The apparatus 
5 also includes a microcontroller or personal computer 414, which may be located in the 
infrared camera 408 itself, attached to the thermography apparatus in some other way, or 
located remotely and accessed via 2-way serial or parallel data communication. A 
measuring instrument 416, such as an ultrasonic thickness gauge, can also be provided to 
conduct thickness measurements or other measurements. Lastly, a user interface 418, 
10 such as an operator keypad or touch screen, is provided on the apparatus to allow 
marking of the defect image and manipulation of the images on the display 412. 

In summary, because the invention uses a distorted live image and a distorted 
defect image to locate subsurface defects, the one-to-one correspondence between the 
two distorted images creates a new frame of reference from which to pinpoint the defect 
15 locations, greatly increasing the accuracy and speed at which subsurface defects can be 
located and marked. The invention also allows complementary NDT/E methods to be 
used in conjunction with the defect image so that the defect image can be annotated or 
can be calibrated so thickness measurements can be obtained directly from the image, 
without requiring prior information from a reference sample or any other source. 
20 Further, the invention can be used with any infrared NDT/E method, independently of 
how the method changes the sample's temperature, generates the defect image, or 
processes the data. 

It should be understood that various alternatives to the embodiments of the 
invention described herein may be employed in practicing the invention. It is intended 
25 that the following claims define the scope of the invention and that the method and 
apparatus within the scope of these claims and their equivalents be covered thereby. 
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CLAIMS 

I CLAIM: 

1. A method for non-destructive evaluation of a sample, comprising the 
5 steps of: 

obtaining a defect image and a live image of the sample, the defect image and the 
live image having a one-to-one correspondence with each other; and 

superimposing one of the defect image and the live image on the other of the 
defect image and the live image. 

10 

2. The method of claim 1, further comprising the steps of: 
locating a defect in the sample via the defect image; 
referencing the defect image according to the located defect; and 
referencing the sample based on the referenced defect image while viewing the 

15 live image and the referenced defect image on a display. 

3. The method of claim 2, wherein the step of referencing the defect image 
includes the step of marking the sample according to the referenced defect image. 

20 4. The method of claim 2, wherein the step of referencing the defect image 

includes the step of measuring a characteristic of the sample at a selected location. 

5. A method for non-destructive evaluation of a sample, comprising the 
steps of: 

25 obtaining a defect image and a live image of the sample, the defect image and the 

live image having a one-to-one correspondence with each other; 
displaying the defect image on a digital display; 

superimposing the defect image onto the live image on the display; and 
referencing the sample while viewing the live image and the defect image on the 
30 display. 
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6. The method of claim 5, wherein the defect image is an infrared image, 
and wherein the defect image and the live image are obtained from an infrared camera. 

7. The method of claim 6, further comprising the steps of: 
changing the temperature of the sample; and 

obtaining at least one defect image over time to locate a defect in the sample. 

8. The method of claim 7, wherein the changing step includes directing a 
heating pulse onto the sample such that the heat is distributed evenly over the sample. 

9. The method of claim 7, wherein the changing step includes directing 
continuous heat onto the sample such that the heat is distributed evenly over the sample. 

10. The method of claim 5, wherein the referencing step includes the steps of: 
measuring a characteristic of the sample at a selected location; and 
annotating the defect image with data obtained from the measuring step. 

1 1 . The method of claim 6, wherein the referencing step includes the steps of: 
measuring a characteristic of the sample at a selected location; and 
annotating the defect image with data obtained from the measuring step. 

12. The method of claim 5, wherein the obtaining, displaying, superimposing 
and referencing steps are automated and conducted in a computer. 

25 13. An apparatus for non-destructive testing/evaluation of a sample, 

comprising: 

a camera that generates a defect image and a live image of the sample; 

a processor coupled with the camera to digitize the defect image and the live 

image; 

30 a display for displaying the digitized defect image and the live image, wherein 

the processor and the display include means for referencing the defect image and 



superimposing one of the defect image and the live image onto the other of the defect 
image and the live image. 

14. The apparatus of claim 13, wherein the processor and the display are 
5 constructed as part of the camera. 

15. The apparatus of claim 13, wherein the camera is an infrared camera, and 
wherein the apparatus further comprises: 

a hood having a reflective interior and an opening for the camera at a back 
10 portion and an open end at the front portion, wherein the sample is disposed in the front 
portion of the hood; and 

at least one heating lamp disposed inside the hood to heat the sample. 

16. The apparatus of claim 15, wherein the hood has a door to allow physical 
15 access to the sample by the user. 

17. A computer readable storage device used to control non-destructive 
testing and evaluation of materials, comprising the steps of: 

obtaining a defect image and a live image of the sample, the defect image and the 
20 live image having a one-to-one correspondence with each other; and 

superimposing one of the defect image and the live image on the other of the 
defect image and the live image. 

18. The computer readable storage device of claim 17, further comprising the 

25 steps of: 

locating a defect in the sample via the defect image; 
referencing the defect image according to the located defects; and 
referencing the sample after the superimposing step while viewing the live image 
and the referenced defect image on a display. 

30 
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19. The computer readable storage device of claim 18, wherein the step of 
referencing the defect image includes the step of marking the sample according to the 
referenced defect image. 

20. The computer readable storage device of claim 18, wherein the step of 
referencing the defect image includes the step of measuring a characteristic of the sample 
at a selected location. 

21 . A computer readable storage device for non-destructive evaluation of a 
sample, comprising the steps of: 

obtaining a defect image and a live image of the sample, the defect image and the 
live image having a one-to-one correspondence with each other; 

displaying the defect image on a digital display; 

superimposing the defect image onto the live image on the display; and 

referencing the sample while viewing the live image and the defect image on the 
display. 

22. The computer readable storage device of claim 2 1 , wherein the defect 
image is an infrared image, and wherein the defect image and the live image are obtained 
from an infrared camera. 

23. The computer readable storage device of claim 22, further comprising the 
steps of: 

changing the temperature of the sample; and 

obtaining at least one defect image over time to locate a defect in the sample. 

24. The computer readable storage device of claim 23, wherein the changing 
step includes directing a heating pulse onto the sample such that the heat is distributed 
evenly over the sample. 
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25. The computer readable storage device of claim 23, wherein the changing 
step includes directing continuous heat onto the sample such that the heat is distributed 
evenly over the sample. 

26. The computer readable storage device of claim 2 1 , wherein the 
referencing step includes the steps of: 

measuring a characteristic of the sample at a selected location; and 
annotating the defect image with data obtained from the measuring step. 

27. The computer readable storage device of claim 22, wherein the 
referencing step includes the steps of: 

measuring a characteristic of the sample at a selected location; and 
annotating the defect image with data obtained from the measuring step. 

28. The computer readable storage device of claim 21, wherein the obtaining, 
displaying, superimposing and referencing steps are automated and conducted in a 
computer. 
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ABSTRACT OF THE DISCLOSURE 
A method an apparatus for non-destructive testing and evaluation of part samples 
includes obtaining a defect image of the sample, displaying the defect image on a 
display, referencing the defect image, such as through marking or annotation, to 
5 highlight locations at which defects or measurements are found, superimposing the 
defect image onto a live image of the part, and physically marking/annotating the part, 
tracing the marks from the defect image onto the physical sample, while viewing the live 
image. Because both the defect image and the live image are viewed through the same 
camera lens and are therefore subject to the same amount of distortion, the actual sample 
10 can be marked exactly according to the marks made in the defect image; there is no need 
to attempt matching a distorted defect image with the physical sample, as has been done 
in the prior art. This one-to-one correspondence between the defect image and the live 
display enables precise marking of the physical sample since the user looks at the 
distorted live image of the sample, not the physical sample itself, during the marking 
15 process. The method is particularly useful for detecting subsurface defects in a sample 
via thermographic techniques. 

R0048519 
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VERIFIED STATEMENT BY INVENTOR CLAIMING 
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As a below named inventor, I hereby declare tfiat I qualify as an independent inventor as 
defined in 37 CFR L9(c) for purposes of paying reduced fees under Section 41(a) and (b) of Title 
35, United States Code, to the Patent and Trademark Office with regard to the invention entitled; 

data integration and marking for infrared 
nondestructive: evaluation 

described in the specification filed on February 25, 1998, Serial No, 6Q/Q75j92Q 



I have not assigned, granted, conveyed or licensed and am under no obligation under 
contract or law to assign, grant, convey or license, any rights in the invention to any person who 
could not be classified as an independent inventor under 37 CFR 1.9(c) if that person had made the 
invention, or to any concern which would not qualify as a small business concern under 37 CFR 
1.9(d) or a nonprofit organization under 37 CFR 1 ,9(e). 

Each person* concern or organization to which I have assigned, granted, conveyed, or 
licensed or am under an obligation under contract or law to assign, grant, convey, or license any 
rights in the Invention is listed below: 

Nona 

I acknowledge the duty to file, in this application or patent, notification of any change in 
Status result in loss of entitlement to small entity status prior to paying, or at the time of paying, the 
earliest of the issue fee or any maintenance fee due after the date on which status as a small entity is 
no longer appropriate. (37 CFR 1.28(b)). 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and farther that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that 
such willful false statements may jeopardize the validity t#the application or any patent to which 
this verified statement is directed. 



Dated: 



i 



>-«j SteVet/M Shepard 

23656 Hunter's Lane 
SouthfieldMI 48034 




09/623071 



OBTAINDEFECT 
IMAGE 


i 


r 


DISPLAYDEFECT 
IMAGE ON COMPUTER 


\ 


f 


REFERENCE 
DEFECTIMAGE 


\ 


r 



100 



-103 



SUPERIMPOSE 
DEFECTIMAGE 
ONTO LIVE IMAGE 



-104 



REFERENCE 
SAMPLE WHILE 
VIEWING LIVE IMAGE 



■106 



Iig-1 



CHANGE TEMPERATURE 
OF SAMPLE 


\ 


r 



OBTAINAT 
LEASTDEFECT 
IMAGE 



TRANSFERDEFECT 
IMAGE TO COMPUTER 
DISPLAY 



1 


f 


MAM OF ANNOTATE 
DEFECTIMAGE 







SUPERIMPOSE 
MARKED DEFECT 
IMAGE ONTO LIVE 
IMAGE 



TRANSFERDEFECT 
IMAGEMARKINGS 
ANNOTATIONTO 
SAMPLE WHILE 
VIEWING LIVE IMAGE 



M-2 




MM. Jyg-JB 




09/623071 




THERMAL kWE IMAGING 



Please typo a plus sign in this box: 



FAX NO. : 248 414 3764 

CD 



Jan. 12 2001 05:28PM P2 



PTO/SB/01 (3-97) 
Approved for u^e through 6/30/98. OMB 0651-0032 
Patent and Trademark Office; US DEPARTMENT OF COMMERCh 









DECLARATION FOR 
UTILITY OR DESIGN 
PATENT APPLICATION 

Q Declaration [3 Declaration 
Submitted with or Submitted after 
initial filing initial filing 


Attorney Docket No. 


64631-031 


First Named Inventor 


Steve M. Shepard 


comple: 


TE IF KNOWN 


" Application No. 




Filing Date 


08/25/00 


Group Art Unit 




Examiner Name 









As a below named inventor, I hereby declare that: 

My residence, post office address, and citi2enship are as stated below next to my name. 

1 believe I am the original, first and sole inventor (only if one name is listed below) or an original, first and joint 
inventor (if plural names are listed below) of the subject matter which is claimed and for which a patent is sought 
on the invention entitled: 



Data Integration and Registration Method and Apparatus for 
Non-Destructive Evaluation of Materials 



the specification of which 

□ 



(Title of the Invention) 



is attached hereto 
or 

was filed on February 25, 1999, as United States Application Number or PCT International Application 
Number: PCT/US99/04206 and was amended on (if applicable), 

I hereby state that I have reviewed and understand the contents of the above identified specification, including the 
claims, as amended by any amendment specifically referred to above. 

1 acknowledge the duty to disclose information which is material to patentability as defined in Title 37, Code of 
Federal Regulations § 1.56. 



Pnor foreign ApptlcMion Numbcr(s) 


Country 


Foreign Filing Duic 
(MM/DD/YY) 


Priority 
Not Claimed 


Certified Copy Attached 

vas HO 










■■ d o 
□ □ 
n □ 



















I hereby claim foreign priority benefits under Title 35, United States Code § 11 9 (a)-(d) of any foreign application(s) 
for patent or inventor's certificate, or § 365(a) of any PCT international application which designated at least one 
country other than the United States of America, listed below and have also identified below, by checking the box, 
any foreign application for patent or inventor's certificate, or of any PCT international application having a filing 



O Additional foreign application numbers are luted on a iupplemcnUl priority data sheet P10/SB/92U attached hcroto 



I hereby claim the benefit under Title 35, United Stales Code §1 19(c) of any United States provisional application^) listed below. 



Application Number (s) 



60/075,920 



Filing Date (MM/DD/YY) 



02/25/98 



L3 Additional provisional application 

numbers are listed on a supplemental 
priority data sheet PTO/SB/02B 
attached hereto. 



FROM : THERMAL UfiUE IMAGING 



i 



Pic wo type u plus bign in thu box: 



FAX NO. : 248 414 3764 



ED 



Jan. 12 2001 05:29PM P3 



PTO/SB/Q1 (3-97) 
Approved tor uac through 6730/98 OMB 065 i -0032 
Patent and Trademark Olflce; US DfJARTMfiNT OF COMMERCE 
Under the Paperwork Reduction Act of I 935, no persons, ara rgquired to respond to a collection of Information uufcfo it display * valid OMB control number 



DECLARATION - Utility Or Design Patent Application 



[ hereby claim the benefit under Title 35, Umted .Stales Code §120 of any United States iipplication(n) oi any PCT International application donating the 
United States of America, listed below and. insofar ui the subject matter of each of the cUtms of this application is not disclosed tn the prior United States 
or PCT International application in the matter provided by ther first paragruph of Title 35, United State* Code §112, 1 acknowledge the duty to dticlosc 
informulioft which U material to patentability a* defined in *Tulc 37, Code of Federal RcguIaUons J 1,56 which became available between ihe filing dure of 
fhtf prior jppliciilion and the national or PCT international filing d<ttc of this application. 



U.S Parent Application 
Number 



PCT Parent 
Number 



Parent Filing Date 
(MM/DD/YYYY) 



Parent Patent Number 
Of applicable) 



Additional U.S or PCT international application numbers arc on a supplemental priority data sheet PTO/SBf02B attached hereto 



As & named inventor, I hereby appoint the following regtMercJ practitioners) to prosecute this application and to tr&m&ct ul] buvine!»» m the Patent and Trademark 

oin« connoted therewith. H Customer Number 01 0291 



Q Registered prictmoner(i) namc/regijiraiion numbor lUteU below 



Number Bay Code 
Label Here 



Name 



til 



Ill 

III 



JOSEPH V. COPPOLA, SR. 



Registration 
Number 



33373 



Name 



Registration 
Number 



E] Additional rc&mcrcd practitioners) named on supplemental ftc^mcrrd Practilioftcr Information uheet PTO/SM)2C aUachod hereto. 



Direct all correspondence to | 



Customer Number 
or Bar Code Label 



Name 



Address 



City, State, Zip 



Country 



010291 



or O Correspondence Address below 



Joseph V, Coppola, Sr., Rapkr, Fishman & Giuuer, PLLC 



39533 Woodward Avenue, Suite 140 



Bloomficld Hills, Michigan 43304 



US 



Telephone | 248-594-0650 



Fax 



248-594-0610 



I hereby declare that all stalemftnta rnude herein of my own knowledge arc Irua and that all itatemcnfcf made on information and belief arc believed to b« 
true; *ind further that these (Utemcntt were irude with the knoW^e thai willful fuUe statements and the like so fnado ure pumsh&Ue by fm« or 
imprisonment, or both, under .Section 1001 of Title 1% of the United Stales Code and that such willful falac statements, may jeopardize ihe validity of the 
aojahouio^ 



N^rn^of Sole or First Inventor 



V 



Given Name (fust and middle [if any]) 



] D A petition has been filed for this unsigned invenfor 




Additional inventors are being named on the ^ ^ supplemental additional inventor^) shcct(s) PTQ/SB/02A attached hereto. 

Name of "inventor j D A petition has been filed for this unsigned inventor 



Given Name (first and middle f if any)) 



Inventor's Signature 



Residence; City 



Post Office Addrc&s 

ca^ 



Family Name or Surname 



State 



Country 



Dated 



Citizenship 



State 



Zip 



Country 



